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ABSTRACT 



This paper discusses objectives associated with the 
training of education students in statistical techniques. A 
historical study shows misconceptions in the areas of evaluation and 
measurement. Further, the typical preparation in quantitative 
methodology provides insufficient experience with analysis of 
problems and development of hypotheses. Objectives presented included 
identification of researchable problems, development of hypotheses, 
selection and application of guantitative techniques, and 
interpretation of the results. It is proposed that the instructional 
program should be viewed not as courses in statistics, but as a 
sequence of experiences in evaluation. Some suggestions are made 
regarding sources of textual materials. This paper was presented at 
the annual meeting of the American Educational Research Association, 
New York, Februrary, 1971. [Not available in hardcopy due to marginal 
legibility of original document. ] (RS) 
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Abstract 

Analysis of the current needs of students, contemporary demands upon 
social scientists, advances in statistics, and in the philosophy and 
psychology of education, yields two directions in which the teaching of 
statistics must move. 1) The integration of statistics with other 
aspects of the evaluation process, from the identification of research 
problems, through the interpretation of data, and 2) development of the 
appreciation of statistics as a refined and consistent symbolic system, 
offering a large variety of analysis options to the researcher. 

Suggested objectives and research findings concerning instructional 
techniques are provided, to aid achievement of "mastery" of the two 
broad goals. 
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UP DATING INSTRUCTION IN STATISTICS 1 
Jeremy D. Finn 

2 

Ontario Institute for Studies in Education 

In 1933, under the direction of Helen Walker, Mr. Ralph Brown 
completed and published his dissertation, - Mathematical Difficulties of 
Students of Educational Statistics. Brown identified and analyzed the 
problems specific to statistics courses in Education. He concluded his 
thesis with a series of suggestions for alleviating the situation -- the 
imposition of mathematical prerequisites, the grouping of students by 
background experience, the elimination of certain topics, providing 
students with the option of computational drill and/or workbooks, and 
the standardization of statistical symbolism. ' Now that we have allowed 
40 years for these suggestions to be successfully implemented, it is 
undoubtedly safe to assume that the problem is no longer with us, and 
is not worthy of further discussion. 

In fact, the problems analyzed by Crown have probably not been 
significantly reduced over four decades.- There has been a change 
however, In social scientists' dispositions about the role of quantita- 
tive techniques in the research process. I would like to analyze this 
role as it is viewed currently, and offer suggestions as to the 
implications for instruction in quantitative methodology. 




Paper presented at the annual meeting of the Ancrlcan Educational 
Research Association, New York, February, 1971. I am indebted to 
Merlin Wahl Strom and Joel Weiss for their reactions to an earlier draft 
of this paper. 
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On leave from State University of New-lork at buffalo. 



What have been our ''objectives 1 ' in teaching statistical methodology 
to students of educational processes? On one level, they are the 
objectives formally, or informally, developed by an instructor for a 
given class and semester. He assert that the students should "know, 
comprehend, apply, analyze, synthesize, and evaluate" the "specifics, 
ways and means of dealing witii specifics, and universals and abstractions" 
of selected material from the statistical domain. 

On another level however, we may ask why it is that we impose these 
"ends" on large numbers of students in Education programs. Here we 
discover some of our more basic assumptions. First, we have come to 
value research, and/or evaluation, as a means for gaining knowledge and 
making decisions essential to progress in education. And second, we 
have noted that the quantification of observations, and the treatment 
of quantified responses, is often a means to facilitate valid research 
or evaluation outcomes. 

Let me describe what is meant here by the processes "research", 
and "evaluation," In the strictest sense, evaluation Is the judgment 
of worth, value, or "goodness" of some object, event, or idea. Thus 
in Education we "evaluate" v.’hen we choose one set of objectives over 
another, when we choose a set of learning experiences for our pupils, 
and when we decide to study particular characteristics of pupils, 
teachers, environments, processes, and outcomes. That is, in so 
doing, wo are delineating that which we think has sufficient value to 
v/arrant our extended efforts in thought and assessment. Further, we 
evaluate when ue select a "sample" of observations for study, and 
when we select measuring Instruments. [I would note parenthetically 
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however, that testing or '’measurement" — the assignment of numbers to 
individuals according to their responses to a given set of stimuli — 
is not in itself evaluation]. Judgments of the relative or absolute 
adequacy of the results obtained, according to a set of criteria, and 
decisions based upon those results, are also evaluations. 

The term evaluation does not in itself imply the use of quantita- 
tive procedures, nor does it even imply the use of accurate and reliable 
measurements. Stake (1970), offers an excellent introduction to think- 
ing about less precise evaluations in his Review of Educational 
Research chapter, "Objectives, priorities, and -other judgment data." 
Nevertheless, the evaluation paradigm uitli which vie have greatest 
familiarity, involves the formal measurement of a sample of "subjects," 
and the subsequent mathemai ical summary and analysis of the outcomes. 
This special case of evaluation, usually termed "research," is more 
accurately termed "quantitative evaluation." Thus a distinction is 
also dravm with non-quantitative research, as in the use of resource or 
historical material to discover problem solutions. 

One of the most concise and to-the-poir.t treatments of the design 
and conduct of quantitative evaluation, is provided by W. T. Federer in 
tiie introductory chapter of Experimental Design (1955). The process 
may be conceived as having eight facets (adapted from Federer). 
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1. Formulation of questions to be answered, 

2. A critical and logical analysis of the problems 
raised. 

3. Formulation of expectations, or hypotheses. 

4. Selection of a procedure for research. 

5. Execution of the research procedures. 

6. A complete analysis of the outcomes. 
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7. Interpretation of results irj light of the hypotheses 
tested and the original questions. 

8. Preparation of a complete, correct, and readable report 
of the evaluation (dissemination of findings). 

Such an outline allows us to examine several common misconceptions. 
The first of these is that quantitative procedures are the only valid 
procedures for conducting "evaluation." The second, and one of 
perhaps greater concern, is the equating of measurement and statistics, 
with the entire evaluation process. A recent unpublished paper 
entitled "Training an educational researcher " (Timm), provides a 
program outline and reading list for the research trainee. The out- 
line and list contain 4 books on "basic statistics," 4 on "correlation 
and regression analysis," 5 on "analysis of variance," 4 on "non- 
parametric statistics," 4 on "design of experiments," 1 on "survey 
sampling," 7 on "multivariate analysis," 4 on "matrix algebra," 4 on 
"measurement-scaling," and 4 on "factor analysis," suggests seminars on 
Bayesian statistics, sequential analyses, Monte Carlo methods, 
stochastic models, time serins analysis, and power analysis, and con- 
cludes with the comment "in addition to the above, the student might 
also find it advantageous to pursue a substantive area of interest." 

This outline surely is extreme in Its neglect of six of the eight 
facets of evaluation listed by Federer. It is not Inconsistent with 

the typical offering of two to three semesters of statistics to 

graduate education students, and no formal instruction In the other, 
perhaps more crucial steps, of the evaluation process. As we would 

expect, the emphases are further reflected In attitudes carried 

beyond the graduate school years. To the extent that research In 
education 1$ not helpful in solving immediate and obviously- pressing 
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problems in today's schools, it is not our use or over-use of 
statistical methods which is at fault, but instead insufficient thought 
given to a) the problems worthy of consideration, and b) multiple hypo- 
theses and educational outcomes. 

What then should our objectives for courses in evaluation methodo- 
logy Include? 

1. The student should develop an awareness of problems 
in Education, and be able to apply to them, 
criteria of problem significance. 

2. The student should be able to identify the aspects 
of educational problems which are amenable to 
solution through quantitative evaluation. He 
should be. able to state or restate such problems 

. in basic, definable terms. He should be able to 
construct a "logical chain' 1 between educational 
practice, theory about human behavior, and specific 
evaluation needs. 

3. The student should be able to derive testable 
hypotheses from theory about human behavior and 
prior research. 

4. The student should value quantitative techniques as 
means for obtaining valid and objective tests of 
his research hypotheses. 

5. Tho student should be able to select appropriate 
hypothesis-testing devices, and to apply them to 
the particular situation. He should be able to 
select and utilize those descriptive devices which 
will clearly, and most simply, provide elucidation 
of the reasons why the major hypotheses are, or are 
not, supported. 

6. The student should be able to Incorporate both the 
quantitative outcomes, as well as the theorectical 
starting points, in deriving interpretations and 
generalizations from the analysis, for the original 
problem situation. 

These are complex objectives, especially so when we realize that 




the "students" are not only our students, but ourselves as well. It 
is thus necessary to ask about structuring a program to promote 
attainment of these objectives. This is nei-.her a small nor unworthy 



The instructional sequence should be viewed not as a course or two 
in statistics, but as a sequence of experiences in evaluation. Probably 
the first and most important unit involves the consideration of problems 
in Education, as. drawn from the personal experiences of the participants, 
from the media, and from readings. Together with the instructor, the 
students should have direct individual and group observations of pupils 
in class situations, and of teachers and administrators. In addition, 
reference to reports of recent observational studies (Jackson, 1963; 

Smith and Geoffrey, 1963), can play a major role in the identification 
and formulation of significant researchable issues. 

The question of problem significance is a most difficult one, and 
one which is often avoided. Class discussion of the relative 
importances of Education problems and outcomes, and of criteria for 
making such judgments, will at least sensitize each student to the need 
for consideration of the significance issue. While scientists often 
disclaim competence or interest in making such value judgments, the 
recent arguments of Scriven (1966, 1970), Pruzek, and others that we 
cannot afford not to consider the issue of significance, merit 
consideration. 

The student should repeatedly be required to select aspects of the 
problem situation, to formalize them in researchable terms, and finally 
to state several hypotheses about the relationships among the major 
constructs. We often profess no prior knowledge from which to antici- 
pate the outcomes of our studies. Frequently, probing of the 
investigator's mind and reference to the findings and theory of others, 
yield surprisingly strong expectations. 
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When the student encounters his courses in quantitative methods, 
and in particular, statistics, he will thus have a starting point from 
which to assess the utility of the techniques presented. Statistical 
techniques in common use are rapidly increasing in nunber and 
complexity. Further, the consideration of '’real-life" and thus very 
complex problems, are likely to require complex models for analysis. 

To avoid occupying the student's entire graduate career with study of 
these models, we must increase our efficiency in their presentation. 

As an alternative to allocating one course unit each, to the topics 
"one-way fixed effects analysis of variance, two-way fixed analysis of 
variance, three-way fixed effects analysis of variance, one-way random 
effects analysis of variance, simple regression analysis, two- 
predictor regression analysis," and so on, it behooves us to consider 
such general models as the multivariate general linear (and nonlinear) 
model, general covariance structure models (Joreskog, 1970), randomi- 
zation models, general path analysis models, and others. From these, 
the students, in roughly the same period of time, can learn to 
construct all the simpler analyses as special cases, plus an entire 
set of more complex models, even beyond those presented by Kays (1963). 
Professors of such courses may, In turn, review the newer material at 
AERA presessions, and special workshops such as those offered at OISE 
and elsewhere. While this may seem a great deal of effort, we must 
ask whether the ultimate "pay-off" will make it worthwhile. 

There is currently a materials problem as well. While there are 
few formal texts to assist us with the newer topics (Oraper and 
Smith, 1967; Mendenhall, 1968), more will be soon forthcoming. In 
the interim, we may utilize less-formal publications usually available 
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from authors-to-be (e.g. Ooreskog, McDonald, Pruzek, Bock, Woodson, 

Timm, Finn). Semi-formal distribution agencies, su^h as The University 
of Chicago Statistical Laboratory, The Thurstone Psychometric Laboratory, 
and others, are likely to be of much assistance. 

What about particular concepts within the statistics courses? 

First, elementary concepts of random variables, distributions of random 
variables, and descriptive functions, may be achieved through a 
"mastery learning" approach, as outlined by Bloom (1968). Constant 
feedback, student v/ork groups, and a plethora of remedial material, 
have effectively promoted mastery in elementary quantitative methods 
courses. 

In the more general discussions of model fitting, these themes 
should be repeatedly stressed: 

1. The distinctions betvreen "estimation" and "hypothesis . 
testing." Both are essential. 

2. The value of multivariate approaches, botii in thinking 
about research problems, and in analysis. 

3. The distinctions between experiments and "comparative 
studies," the disadvantages as well as the strengths 
of each. 

4. The utility and strengths of planned hypotheses and 
related devices, such as the estimation of planned 
contrasts among model parameters. 

Finally, provide the students with nimcrous opportunities to analyze 
real data, to probe the data for hidden meanings and findings, and 
especially to make recommendations for educational practice from the 
outcomes. 

What I am suggesting then, is not an extended sequence of 
statistics courses for students of educational problems. It is 
Instead an integration of quantitative analysis techniques into the 
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research and evaluation processes, at an early point in student's career. 
Research in Education can be "relevant," and quantitative techniques can 
play a part in making it so. In order to achieve this, we must expect 
much of ourselves, in terms of spending a significant .mount of time 
In the systematic foimulation of important Education problems. If we, 
and our students, view the ultimate outcome as worthy, they may expect 
and possibly attain a higher level of statistical competence than at 
present. In addition, we may be fortunate enough to achieve parallel 
attitude changes concerning the utility of evaluation. To maximize the 
likelihood of such changes, we may employ the achievement-supporting 
nature of 1) teacher Interest and involvement, and £) pupils' and 
teachers' expectations. We must also expect much of our colleagues. 

The students' attitudes reinforced from multiple sources are those 
most likely to develop. 

By reconsidering the role of statistics in social sciences, I hope 
that we can eliminate discussions of the sort I overheard several 
months ago. A student in Education at an institution in sane distant 
country, asked a professor to sit on his dissertaion committee. When 
queried about the topic or problem area; the student replied, "I 
haven't decided on a topic yet, but I'll probably do a two-way analysis 
of variance." 
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